Repository navigation
feat(langchain): add the langchain and langgraph callback handler - #217
Merged
Merged
Conversation
TelemetryDevCallbackHandler records the runs of LangChain.js and LangGraph.js as spans of @telemetry-dev/sdk: graphs as invoke_agent spans, nodes and chains, model calls with usage and bound tools, tools, and retrievers. Each span has langchain.run.id, and the ingest uses it to classify the span as framework langchain.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LangChain.js and LangGraph.js apps can now send traces to telemetry.dev through our own package. At this time the LangChain guide covers only Python, through OpenLLMetry, and a TypeScript app must use a third-party instrumentation.
The handler is a LangChain callback handler on
@telemetry-dev/sdk, as the provider wrappers are. Thus theinit()options (capture, mask, batching),propagateAttributes(), and the spans ofobserve()operate on the LangChain spans too. A LangGraph run becomes aninvoke_agentspan, and each node becomes a span under it. Each model call becomes achat <model>span with usage, finish reason, and the type and name of the bound tools ingen_ai.tool.definitions. Each tool becomes anexecute_toolspan. LangGraph copiesconfigurable.thread_idinto the metadata of each run, and the handler uses it as the session.The handler does not record runs that LangChain tags
langsmith:hidden, such as__start__. Their child runs go under the nearest recorded run.The handler sets
awaitHandlers. Without it, LangChain calls handlers from a shared background queue. There the active context is not the context of the caller, and concurrent runs go under the wrong span of the app.The declaration of
handleChainStartin@langchain/corelists the parameters in a different order from the call that LangChain makes. The handler uses the order of the call, and a comment inhandler.tstells this.If the state of a graph has a
messageslist, the output of the graph span is the last message. The trace list shows the first 4,000 characters of the root output, and a full message state puts the final answer after that limit.Each span has
langchain.run.id. With telemetry-dev/telemetry.dev#1113, telemetry.dev shows these spans as frameworklangchain. An older ingest shows them asotel.The lockfile adds
@langchain/core1.3.4 and@langchain/langgraph1.7.0 as dev dependencies. pnpm then shares their@cfworker/json-schemaandws8.21.3 with the@modelcontextprotocol/sdkand@google/genaientries, and the peer suffixes of those entries change.The LangGraph quickstart agent from the LangChain docs, with Claude Sonnet 4.6, in telemetry.dev:
Related: telemetry-dev/telemetry.dev#1113
Follow-up for the docs page and the setup flow: https://github.com/telemetry-dev/telemetry.dev/issues/1114